Escort entropies and divergences and related canonical distribution 
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Abstract 



. We discuss two families of two-parameter entropies and divergences, derived from the standard Renyi and Tsallis 

entropies and divergences. These divergences and entropies are found as divergences or entropies of escort dis- 
tributions. Exploiting the nonnegativity of the divergences, we derive the expression of the canonical distribution 
r*j ' associated to the new entropies and a observable given as an escort-mean value. We show that this canonical distri- 

<*S ^ bution extends, and smoothly connects, the results obtained in nonextensive thermodynamics for the standard and 

^ ,Jh generalized mean value constraints, 

-t— > . 
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1. Introduction 



Renyi and Tsallis entropies extend the standard Shannon-Boltzmann entropy, enabling to build generalized 
thermostatistics, that include the standard one as a special case. The thermodynamics derived from Tsallis entropy, 
the nonextensive thermodynamics, has received a high attention and there is a wide variety of applications where 
experiments, numerical results and analytical derivations fairly agree with the new formalisms (lfl. Some physical 
applications of the generalized entropies, - including statistics of cosmic rays, defect turbulence, optical lattices, 
systems with long-range interactions, superstatistical systems, etc., can be found in the recent review J2l and 
references therein. In the extended thermodynamics, it has been found particularly useful to use generalized 
moments J3[|3l. These moments are computed with respect to a deformed version of the density at hands, which 
is called its escort distribution J5l0]. Actually, several type of constraints J3,[3,@] have been used in order to 
derive the canonical distributions: a first type of constraints is expressed as standard linear mean values while 
a second type of constraints is given as generalized escort mean values. In the two cases, the related canonical 
distributions are expressed as a g-Gaussian distribution, but with two opposite exponents, and both solutions reduce 
to a standard Gaussian distribution in the q = 1 case. These (/-Gaussian distributions can exhibit a power-law 
behavior, with a remarkable agreement with experimental data, see for instance lfH [loi[TH] . and references therein. 
These distributions are also analytical solutions of actual physical problems, e.g. jl2lll3i [l4ll . There has been 
numerous discussions regarding the choice of a 'correct' form of the constraints, either as a standard mean or 
an escort-average, and on the connections between the solutions and associated thermodynamics. In particular, 
dualities and equivalences between the two settings have been described. 

This is precisely the context of the present Letter, where we propose a simple connection between these dif- 
ferent formulations and between the related canonical distributions. More specifically, we suggest a simple way 
to combine the originally distinct concepts of entropy and escort distributions into a single two-parameter (a, A)- 
entropy. Then, we propose to look at an associated extended maximum entropy problem. This approach includes 
the two aforementioned formulations as special cases. Exploiting the nonnegativity of the associated divergence, 
we derive the expression of the canonical distribution for an observable given as an escort-mean value. We show 
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that this canonical distribution extends, and smoothly connects, the results obtained in nonextensive thermodynam- 
ics for the standard and generalized mean value constraints. 

We begin by recalling the context and definitions in section [2] Then, the combined escort divergences and 
entropies are introduced and discussed in section [3] The related maximum entropy problem and its solution are 
described in sectionH] Finally, in section|5] we illustrate the results in the case of a two-level system. 



2. Context and definitions 

2.1. Main definitions 

Let us recall that if / and g are two probability densities defined with respect to a common measure //, then for 
a parameter q > 0, called the entropic index, the Tsallis information divergence is defined by 

D q T \f\\g) = — r ( [ f(x)*g(x) x -Hrix) 1 



9-1 

and similarly, the Renyi information divergence is defined by 

D g R) (f\\g) = ^Y lQ g/ /(aOW 1- *^). (D 

provided, in both cases, that the integral is finite. By FHospital's rule, both Tsallis and Renyi information diver- 
gences reduce to the Kullback-Leibler information in the case q = 1. Associated to these divergences, we obtain 
the entropies when g(x) is uniform with respect to /i. For q ^ 1, 

s q [f] = (/ WM*) i) > (2) 

is the Tsallis entropy, and 

H q [f] = j^log J f(x)"dfx(x), (3) 

is the Renyi entropy. In both cases, we have S\ [/] = Hi [/] = —/ / log / dfj,(x), the Shannon entropy, for q = 1 . 
For the Lebesgue measure, the definitions reduces to the standard differential entropies. Finally, in the discrete 
case, the continuous sum is replaced by a discrete one which extends on a subset V of integers, and /i is a counting 
measure, usually taken as uniform. We will also denote 

M q [f] = J fixfA^x) (4) 

the integral of the density raised to the power q. 

The escort distributions used in nonextensive statistics are defined as follows. If f(x) is an univariate proba- 
bility density, then its escort f a (x) of order a, a > 0, is given by 

f (x) fix)a (5) 

provided that M a [/] = J f(x) a d/j,(x) is finite. Given that f a (x) is the escort of f(x), we see that f{x) is itself the 
escort of order 1 /a of f a {x). Accordingly, the (absolute) generalized a-moment of order p is defined by 

m p , a [f] = j Wf a W* - Jf( X )a Mx) ■ (6) 

Of course, standard moments are recovered in the case a = 1. 

In the context of nonextensive statistical mechanics, the parameter a is taken equal to the parameter q of the 
g-entropy. We shall also point out that the 'deformed' information measure like the Renyi entropy ([3]) and the 
escort distribution (f5]l are originally two distinct concepts, as indicated here by the different notations a and q. 
There is an interesting discussion on this point in IU5I1 . It is a contribution of the present paper to indicate that it is 
still possible to adopt two different values for these parameters. 



2.2 The maxim um en tropy problems 
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2.2. The maximum entropy problems 

In nonextensive thermostatistics, the maximum g-entropy distributions, which are identified as g-gaussian dis- 
tributions, can be obtained using a constraint expressed as a classical mean value; but it has also been found 
adequate to use constraints given as escort-mean values, with the same index q as in the entropy. This is discussed 
in H or in QQ]. In the first case, the maximum entropy problem stands as the variational problem 

H<V(m) = max : m p>1 [f] = m and m 0) i[/] = 1} (7) 

which consists in finding a distribution with maximum Renyi entropy on the set of all probability distributions 
with a fixed moment of order p: m p i [/] = m. The value of the maximum entropy obtained for a given moment 
is denoted H q (m) - the use of the square brackets and parenthesis distinguishes between the functions of the state 
and the functions of the observable. Similarly, for a constraint given as an escort-mean value, we have 

H q q \m) = max{H q [f] : m p , q [f] = m and m ,i[/] - 1} . (8) 

Note that although the problems above have been written for the Renyi entropy, the very same problems can 
be written for Tsallis entropy; actually the optimum distributions are identical and the optimum values of both 
maximum entropy problems are easily related. 

In the case of the standard mean constraint, the density that achieves the maximum entropy is given by the 
generalized g-Gaussian 

1 

while in the case of the escort mean constraint, the solution has an exponent with opposite sign 



= — (!-(<? -i)/w)r (9) 



f(x) = ^r(l-(l-q)i3\x\vy + -*. (10) 

In both cases, we used the notation (x) + = max {x, 0} , and j3 is a positive parameter chosen so as to satisfy the 
constraint. The limit case q — > 1 gives f(x) oc cxp(—(3\x\ p ) . The partition functions Z(f3) used to normalize 
f(x) as a density will be given below. 

2.3. Dualities 

There has been a great deal of discussions concerning the relationships between the two problems © and ([8]) 
with standard and escort mean value constraints, and between their solutions (O and 1101 . Dualities between the 
solutions have been introduced: the q — > 2 — q transformation 1 1^, 17] enables to obtain the density dTOb from the 



maximization of the entropy S2- q [f] subject to standard constraints rather than the maximization of S q [f) subject 
to escort generalized constraints. The q — >• 1/q duality rests on the duality between the original density and its 
escort: as already noted, if f q is the escort of order q of f, then / is itself the escort of order 1/q of f q . This has 
been mentioned in 0], the related duality discussed in 1 1_8 ] and in jl$\ . It is very easy to obtain the q — >• 1/q 



duality as the consequence of the symmetry above for escort distributions: it suffices to note that for the Renyi 
entropy, we always have the equality Hi [f q ] — H q [f}. Therefore, noting that we also have m p , q [f] — m Pt i[f q ], 
we see that the problem 

H q "\m) - max : m p , q [f] - m and m ,i[/] = 1} , (11) 

is exactly equivalent to 

Ht*>{m) - max : m p ,i[/J = m andm ,i[/] = l} = H?\m) (12) 

where the maximum entropy is obtained for the density ( fTOb . Therefore, it is always possible to swap the standard 
and generalized constraints, provided that entropic index q is changed into 1/q. We shall also mention that these 
dualities have been linked to combinatorial considerations in 12011. 
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3. The (a, A) divergences and entropies 



In complement to the dualities mentioned above, we will show that there is a continuum of q-Gaussians, 
solutions of an extended maximum entropy problem, that smoothly connects the problems (01 and ^ and their 
solutions (0 and ( TTOb . The basic idea is to mix the concepts of g-entropies and escort distributions into a single 
quantity. This leads us to a simple extension of the Renyi (Tsallis) information divergence and entropy to a two 
parameters case. Interestingly, the generalized (a, A)-Renyi information divergence that emerges in ( fT4b has been 
mentioned 12 lL eq. (49)] as a possible extension of a new class of (a, /3)-divergences, generalizing a family of 
Gamma-divergences recently introduced in B22I1 . 



Definition 1. Given two parameters a and A > and two densities / and g, we call (a, A)-Renyi information 
divergence the standard Renyi information divergence with index q = a/X between the escort distributions like 
(0, of order A, f\ and g\, associated to / and g : 



D^(f\\g) = D^>(f x \\g x ). 



In the developed form, this gives 

D $(f\\9) = 



A 



log 



A-o 



(13) 



(14) 



provided that the different integrals exist. 

Similarly, we call (a, A)-Tsallis information divergence the quantity 



1 



^i T A ) (/H9) = 



provided that the different integrals exist. 



[f f(xy g (x) x - a dfi(x)Y 



[ff(x)^(x)] a [Jg(x)^(x)] 



\-a 



(15) 



These definitions include the Renyi (Tsallis) information divergence Da\f\\g) in the case A = 1. In the 
case A — a, again by l'Hospital's rule, (fT4l i becomes D x \(f\\g) = D(f\\\g\), the Kullback-Leibler divergence 
between the escort distributions of order A, fx and g\, associated to / and g. Finally, in the case a = A = 1, the 
generalized Renyi (Tsallis) information divergence reduces to the standard Kullback-Leibler divergence. 

Associated to this (a, A)-Renyi information divergences, we can also define a two parameters Renyi (Tsallis) 
entropy. 

Definition 2. Given a and A > and a density /, we call (a, A)-Renyi entropy the entropy of order q = a/X of 
the escort fy. 

' f(x) a dKx) 



H a , x [f} = H f [f x ] = 



X — a 



los 



A r 



(16) 



provided that the different integrals exist. Similarly, the (a, A)-Tsallis entropy is given by 



S a , x [f} = Sf[fx} = 



1 



A- 



f(x) a d^x) 



f{x) X d^{x) 



- 1 



(17) 



Again, for A = 1 we get the standard Renyi (Tsallis) entropy of order a, for a = 1 we get the standard Renyi 
(Tsallis) entropy of order A, for a = A it reduces to the Shannon entropy of the escort distribution, and to the 
standard Shannon entropy in the case a = A = 1. 

In the following, we will need the fact that the divergence D^\(f\\g) is always nonnegative: 

Proposition 3. if f and g are two densities such that the involved integrals are finite, with a and A > 0, then 
xiflld) — 0: with equality if and only if f = g. 

Proof. Obviously, once we have realized that D^\(f\\g) in ([Pit is the Renyi (Tsallis) divergence between escort 
distributions, this is a direct consequence of the nonnegativity of the Renyi (Tsallis) divergence, which follows 
from Jensen inequality. Alternatively, this could also be derived from the Holder inequality. □ 
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4. The (a, A) maximum entropy problem 



With the previous definitions at hands, we can now consider the following extended maximum entropy problem: 



H a ,\{m) = max{H a> x[f] : m p>a [f] = m andm ,i[/] = 1} (18) 

which includes the previous problems as particular cases. The usual procedure for handling such variational 
problem is the technique of Lagrangian multipliers, e.g. J3,[Ml- However, even though the objective functional is 
strictly concave (here for a < A), the constraints set is not convex and the uniqueness and nature of the maximum 
can not be guaranteed. The situation is still more involved in the continuous setting considered here which requires 



the results from calculus of variations 11231,12411 . Therefore, we propose to derive the solution as consequence of the 
nonnegativity of D a ,\(f\\g), without recourse to the technique of Lagrange multipliers. We obtain that the density 
f(x) that achieves the maximum in the right hand side of ( TT8l is a generalized Gaussian, as given by the following 
proposition. 

Proposition 4. The density that achieves the maximum of the (a, X)-Renyi or Tsallis entropy, subject to the escort 
mean constraint m Pta [f] = m, is the generalized Gaussian 

G (x) = {^W) {l - {X - a) ^ P) ^ f ° rA ^ a (19) 
IzfeexpH^H ifA = a 

where f3 is a positive parameter. The partition function Z(J3) is given by 



for a > A > a — p 

for\> a (20) 
and if A = a 



z(p) 


2 

P 


Ua 


-A) 




-a) 




i) 



where B(x, y) is the Beta function. Finally, the a-moment of order p is given by 



I ]3pa 

For A > a, the density has a compact support, while for A < a it is defined on the whole real axis and behaves 
as a power distribution for a; — > oo. 

Proof. The expressions of the partition function and of the a-moment are obtained by direct calculation. For the 
main result, let us consider the generalized Gaussian ( fT9l , and denote A(f3) = 1/Z((3) the inverse of the partition 
function. We directly have 

J f a G x f a df,(x) = A{pf- a M a [f] x J (I - (A - a)p\x\n +J £jr ] dKx) (22) 

< A([3) x - a (l - (A - a)f3m Pia [f}) M a [f], (23) 

where we have exhibited the escort distribution f a , where m p a [/] denotes the generalized a-moment, and where 
the inequality in ( 1231 results from the fact that for A > a, the support of (1 — (A — a)[3\x\ p ) + can be smaller than 
the support of /. From ( |23l we also immediately get with / = Gp that 

M\[Gp) = A((3) x ~ a (1 - (A - a)Pm a , p [Gp)) M a [Gp\. (24) 



for A-a^O 
for A = a. 



(21) 
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Therefore, for all distributions / with a given a-moment m PjQ [/] = m and for j3 such that the generalized Gaussian 
has the same moment m Pta [Gp] = m, then the combination of d23T > and (f24t gives 

Finally, the (a, A)-Renyi divergence can be expressed as 



^S(/ll^)=log 



Jf a G x f a df,( 



by the definitions < TT~4T > and ( TToT l of the divergence and entropy. By the nonnegativity of the divergence, we obtain 
that 

H a ,x{G(j) > H a , x (f) 

for all distributions / with a given a-moment m pM [f] = m P:a [Gp] = m, and with equality iff / = Gp. Using the 
(a, A)-Tsallis divergence rather than Renyi's, the same result also directly follows for the (a, A)-Tsallis entropy. □ 

From this result, we see that the problem (TT~8T > and its solution indeed interpolates between the maximum 
g-entropy problem with standard constraint (JTJ and with generalized g-moment constraint ([8]). 

• For a = q and A = 1, the (g, 1)-Renyi entropy is the standard Renyi entropy with index g, whose maximum 
subject to a g-moment constraint is attained for the generalized Gaussian ( fTOb with exponent 1/ (1 — q) . 

• For a = 1 and A = q, the (1, g)-Renyi entropy is the Renyi entropy with index g, whose maximum subject 
to a standard moment constraint is the generalized Gaussian (O with exponent l/(q — 1). 

It shall be also mentioned that we still get a?-) 1/g duality result in this setting. From the definition (1161 . we 
always have the identity H a ^x[f] = H<±. [fx] = Hx [f a ]. In the other hand, the generalized a-moment m Pia [/] can 
be considered as the standard moment of density f a , m pM [f] = m p i[/ a ] but also as the escort moment of order 
a/A of the density fx, i.e. m p ^ a [f} = m p ^ [fx]- Therefore, the problem dT8b can be recasted in the two equivalent 
forms 

H a M m ) = m ax {^A[/a] : rripAfa] = m andm o,i[/] = 1 j (25) 
which is a problem with respect to the density f a under a standard mean constraint, or 

H a ,x{m) = max [fx] : m p a [f x ] = m and m ,i[/] = 1 j (26) 
which is a problem with respect to fx with a generalized moment constraint. 



5. The case of a two-level system 

Finally, we close this paper with the example of a two-level system, with eigenenergies and 1. Although 
the general (a, A) maximum entropy problem (TT~8T > usually do not lead to closed-form expressions for the entropies 



H a ,x( m ), see e -g- 12511 . we still can get an explicit solution for the simple two-level system. For this system, 
the measure at hands charges the two levels, with /j = 5q + 5\ where 5 X denotes the Dirac mass at x. Then, the 
maximum entropy distribution is the discrete distribution with a probability p for the excited state and (1 — p) for 
the ground state, with 

P = — • 

l + (l + ^)~ 
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In turn, the (a, A)-entropy is then given by 



1 



H a AGp] = t log [p a + (1 - p) a ] X [p x + (1 - p) x ] , 

A — CI 

and the internal energy, to, which is computed with respect to the escort distribution of order a is simply 

p a 

TO = - — . 

p a + (1 - p) a 

With a little algebra, it is possible to express p as a function of to: 



P 



ma + (1 — to) a 
and then we can get the expression of the (a, A)-entropy: 



H at x(m) 



a — A 



log to» + (1 — to)" 



From this expression, it is easy to derive an expression of the inverse temperature as the derivative with respect to 
m, or the expression of the related heat capacity. We also see that the subsequent entropy only depends on the ratio 
A/a. Finally, for A = a, l'Hospital's rule gives 

H ata (m) = —to log [to] — (1 — to) log (1 — to) , 

which is the Fermi-Dirac entropy. 



6. Conclusions 

In this Letter, we have suggested a possible extension of the standard q-divergences and entropies by con- 
sidering a combination of the concepts of g-divergences and of escort distributions. This leads to two-parameter 
information measures that recover the classical ones as special cases, and coincide with some recently proposed 



measures 12 1H . Further work should examine the general properties of these information measures. We have intro- 
duced a general maximum entropy problem and derived the expression of the canonical distribution maximizing the 
(a, A)-entropy under a generalized a-moment constraint. This was obtained as a consequence of the nonnegativity 
of the related information divergence, without recourse to the technique of Lagrange multipliers. This canonical 
distribution is a two-parameter version of the generalized q-Gaussian distribution, extending the versatility of this 
distribution. It includes the standard solutions of the nonextensive thermostatistics as particular cases. Actually, 
the proposed generalized maximum entropy problem includes the standard approaches of nonextensive thermo- 
statistics and provides a continuum of problems and solutions between them. In addition, this approach suggests 
that it is possible to adopt different indexes for the g-entropy and for the escort-mean constraint. 
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